All Questions
Tagged with seleniumweb-scraping
71 questions
3votes
1answer
83views
Multi-Page Web Scraping Code Using Selenium with Multithreading
I have written a web scraping script using Selenium to crawl blog content from multiple URLs. The script processes URLs in batches of 1000 and uses multithreading with the ThreadPoolExecutor to ...
- 31
3votes
1answer
86views
Scraping website with Python and Selenium to collect data from dynamic website
Summary: The code scrapes the website and collects the data to store it in CSV. It also downloads selected information that is available for download in PDF format. The details and the entire code are ...
- 269
2votes
1answer
66views
A selenium web scraper to package NBA data
I'm building a selenium web scraper for basketball-reference.com that takes a player name and returns data in either a JSON format or Pandas DataFrame object. The class in question is one of many that ...
0votes
1answer
110views
Poetry Web Scraping in Python [closed]
I have a script that obtains urls that lead to a specific poem. This code current works and uses multiprocessing pools. I currently am getting restricted or blocked by some way from the website that I ...
1vote
1answer
223views
Selenium/Beautifulsoup web scrape of Grainger
This is my first time writing a web scrape using Selenium and Beautifulsoup. The website I'm scraping is https://www.grainger.com/ and I have it pulling a specific set of SKUs stored in an Excel file. ...
2votes
1answer
57views
scraping pollution data tables from government website
i made this python program to scrape and save the daily maximum pollution values in mexico city, the data will be used in machine learning but i wonder what could be improved: ...
3votes
1answer
200views
Scrape PokeDex and display in tkinter
Hi I am new here and I just completed my first working version of a pokedex app with a GUI using tkinter. I used selenium to scrape the data from pokemondb.net, and then used pandas to clean up the ...
3votes
1answer
185views
Crawler/scraper for soccer match results
I wrote this code some time ago as part of the web-scraping learning. Every now and then I find mistakes in it, as well as I have doubts. Feedback please, is this code compliant with the common best ...
1vote
0answers
249views
Web scraper for pokemon prices using Selenium
I made this scraper to pull the prices of pokemon cards off tcgcollector, using a csv file from the same site. Because for some reason they don't export price. I'm looking for any kind of noobie ...
- 11
1vote
1answer
377views
Selenium web scraper to scrape Apple tweets of the last 3 years
I want to use my Web Scraper to get all tweets about Apple up to a date I specify. At the moment I'm scraping the tweets of today, or the last few days. However, my goal is to scrape all tweets from ...
- 11
3votes
0answers
88views
Scraping housing dataset row by row from Estate Agent's website
I have written web scraping code with selenium with python3 for a dynamic site (it was my first project that is why I started with selenium). In my code I extracted the values and created DataFrame. ...
4votes
2answers
346views
Scraping campsite availability from a webpage using vba with selenium
I wrote some code to extract the information from a table, but it takes an extremely long time. The table is in the format of a calendar. I need the information on an Excel sheet with column 1 as the ...
- 41
1vote
1answer
371views
Web-scrape bonds information from nasdaqomxnordic
I'm a newbie getting into web scrapers. I've made something that works, it takes 3.2 hours to complete job and randomly have 10 lines blank each time I run this job. Help is much appreciated! ...
- 13
4votes
2answers
603views
Web scraping data.cdc.gov for COVID-19 Data with Selenium in Python
I'm attempting to scrape data.cdc.gov for their COVID-19 information on cases and deaths. The problem that I'm having is that the code seems to be very inefficient. It takes an extremely long time for ...
- 55
2votes
0answers
604views
My first vba-selenium code
I have been obtaining financial information on an annuity I have which is invested in multiple funds which are proprietary to the insurance company. The asset value is not available other than on ...
